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ABSTRACT 

^^ I The deep near-infrared luminosity function of AC 118, a cluster of galaxies at z — 0.3, is presented. 

AC 118 is a bimodal cluster, as evidenced both by our near-infrared images of lensed galaxies, by public 
X-ray Rosat images and by the spatial distribution of bright galaxies. Taking advantage of the extension 
and depth of our data, which sample an almost unexplored region in the depth vs. observed area 
Mh' diagram, we derive the luminosity function (LF), down to the dwarf regime {M* + 5), computed in 

.^ I several cluster portions. The overall LF, computed on a 2.66 Mpc^ areas {Hq = 50 km s~^ Mpc^^), 

has an intermediate slope (a — —1.2). However, the LF parameters depend on the surveyed cluster 
^^ ' region: the central concentration has 2.6^^^ times more bright galaxies and 5.3^2 3 times less dwarfs 

Cn , per typical galaxy than the outer region, which includes galaxies at an average projected distance of 

~ 580 kpc (errors are quoted at the 99.9 % confidence level). The LF in the secondary AC 118 clump is 
>^ ^ intermediate between the central and outer one. In other words, the near-infrared AC 118 LF steepens 

J^ . going from high to low density regions. At an average clustercentric distance of ~ 580 kpc, the AC 118 

-^ ' LF is statistically indistinguishable from the LF of field galaxies at similar redshift, thus suggesting that 

pf-j , the hostile cluster environment plays a minor role in shaping the LF at large clustercentric distances, 

Q>^ ■ while it strongly affects the LF at higher galaxy density. 
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1. INTRODUCTION 

The luminosity function (LF hereafter), i.e. the number 
density of galaxies having a given luminosity, is critical 
to many observational and theoretical problems (see e.g. 
Binggeli, Sandage & Tammann 1988). For example, it is 
needed to interpret galaxy counts (e.g. Driver & Phillipps 
1996), to compute the spatial covariance from the angu- 
lar correlation function (e.g. Koo & Kron 1992), and to 
constrain the primordial power spectrum of density fluctu- 
ations (e.g. Ostriker 1993). From an observational point 
of view, the LF is the natural "weight" of several quanti- 
ties since most statistical quantities are weighted on the 
relative number of objects in each luminosity bin. For 
example, the fraction of blue galaxies, which is the ba- 
sic measure used for measuring the Butcher-Oemler effect 
(Butcher & Oemler 1978, 1984), is a (normalized) convo- 
lution, over the LF, of the color distribution at a given 
luminosity. Furthermore, due to the role played by lumi- 
nosity in the inclusion of the objects in the studied sample 
(faint objects are often excluded or under-represented), 
the knowledge of the LF is fundamental to the calculation 
of the selection function and it is needed to derive the ac- 
tual galaxy properties from the measured quantities. For 
instance, the large increase of the brightness of galaxies 
with z may be partially due to a ^-dependent sampling of 
the LF (Simard et al. 1999). 

Due to the central role played in many astrophysical 
problems, the optical LF of galaxies in clusters has been 
extensively studied (e.g., to cite just a few papers dealing 
with large numbers of clusters, Gaidos 1997; Valotto et al. 
1997; Lumsden et al. 1997; Garilli, Maccagni & Andreon, 
1999). However, not all wavebands carry the same infor- 
mation: optical filters are, with respect to near-infrared 
ones, more affected by short lived starburst events and 
thus better tracers of the metal production rate but worse 
tracers of the underlying stellar mass (Bruzual & Chariot 
1993). It is therefore valuable to extend the LF measure to 
other bands. Near-infrared filters are useful in several as- 
pects: with respect to optical filters they are less affected 
by internal and Galactic absorption, and their differential 
(type to type) K-corrections are quite small (up to z ^ 1) 
thus not altering the cluster content just because of an 
observational effect. 

The present knowledge of the cluster near-infrared LF 
is quite fragmentary and it is limited either to small areas 
at moderate depth (e.g. Barger et al. 1996; Trentham 
& Mobasher 1998; Aragon-Salamanca et al 1993) or to 
relatively large areas, but at bright luminosities (e.g. de 
Propris et al. 1999; Barger et al. 1998). This implies, 
for instance, that dwarfs galaxies are not sampled out- 
side the cluster core. As a first step to overcome these 
drawbacks, we present in this paper the LF of a cluster 
of galaxies, AC 118, computed over one of the largest ar- 
eas (in Mpc^) and to a depth only rarely achieved for any 
cluster of galaxies. 

AC 118, also known as Abell 2744 (AbcU 1958; z = 
0.308), is one of the most observed clusters at intermedi- 
ate redshift. It was first studied by Couch & Newell (1984) 
on photographic plates, then low dispersion spectroscopic 
data were acquired (Couch & Sharpies 1987; Barger et al. 
1996) . For the very central region of the cluster (r <^ 90" ) 
Hubble Space Telescope images have been used for mor- 



phological studies (Couch et al. 1998; Barger et al. 1998) 
and mass determination through gravitational lensing ex- 
periments (Small et al. 1997). Near-infrared {K'-hand) 
photometry (Barger et al. 1996) of the very central region 
of this cluster at intermediate depth and with a coarse 
angular resolution {FHWM ~ 1.7 arcsec) is also avail- 
able. Barger et al. (1996) present U and / photometry 
for the brightest K' sources. AC 118 shows an excess of 
blue galaxies (Couch & Newell 1984), commonly known 
as Butcher-Oemler effect (Butcher & Oemler 1978, 1984) 
and it is one of the clusters that triggered the discussion 
on whether the star formation rate increases or decreases 
during the galaxy infall in the cluster (Barger et al. 1996; 
Balogh et al. 1997, 1998). 

The paper is organized as follows: in the next section, 
the data and the data reduction are presented. In Sec- 
tion 3, the AC 118 LF is derived in a few cluster regions. 
The discussion and a summary arc presented in Section 4. 

In this paper we assume Hq — 50 km s~^ Mpc~^ and 
qo = 0.5. If Oa = 0.7, JIm == 0.3, AC118 galaxies would be 
^ 0.3 brighter. The main result of this paper, a cluster- 
centric dependence of the LF, is independent on the cos- 
mological parameters because all compared galaxies are at 
the same distance from us. 

2. DATA AND DATA REDUCTION 

2.1. AC 118 Observations 

AC 118 observations were carried out at the 3.5 m NTT 
with SOFI (Moorwood, Cuby and Lidman, 1998) on Oc- 
tober, 11, 1998, in the frame of an observational program 
aimed at deriving the Fundamental Plane at z ~ 0.3. 
SOFI is equipped with a 1024 x 1024 pixel Rockwell 
"Hawaii" array. In its large field mode the pixel size is 
0.292 arcsec and the field of view 5x5 arcmin. The field 
was observed in the near-infrared Kg passband (Ac = 2.2/x; 
AA ^ 0.3/i) during a photometric night with good seeing 
{FWHM < 0.8 arcsec). The total useful exposure time 
is 15900 s, resulting from the coaddition of many short 
jittered exposures. Jittering was controlled by the auto- 
matic jitter template (described in the SOFI User Man- 
ual) which produces a set of dithered frames. Offsets were 
randomly generated within a box of 40 x 40 arcsec cen- 
tered on the cluster center {a : 00 14 19, 5 : —30 23 18, 
J2000). The effective exposure time of each individual 
AC 118 frame was 1 min in Ks, resulting from the average 
of 6 exposures of 10 s each. Photometric calibration has 
been obtained by observing a few standard stars, inter- 
spersed with AC 118 observations, taken from the list of 
Infrared NICMOS Standard Stars now published in Pers- 
son et al. (1998). The effective exposure time for standard 
stars was 100 s, given by exposures taken at five different 
array locations, each one being the average of 10 integra- 
tions 2 s long. Fig. 1 shows the final Kg image of AC 118. 

2.1.1. Reset, gain and illumination corrections 

All images have been fiat-fielded by flaton-flatoff. All 
pixels whose gain differs from the average by more than 30 
% have been flagged and not used in the image combining. 

Hawaii chips have a special feature: after reset, the zero- 
level is not constant over the field and its amplitude de- 
pends on the total charge collected in the previous expo- 
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sure. The variation of the zero-level is due to the fact that 
rows have not the same reset voltage after reset because 
resetting an Hawaii chip is a power demanding operation 
(Mackay et al. 1998). This pattern is additive and equal 
for all pixels in the same row since they are reset in paral- 
lel. This pattern is unimportant for science images, whose 
background is anyway spatially not uniform and variable 
in time. However, this spatial pattern and its time vari- 
ation are important in the determination of the gain cor- 
rection (i.e. of the flaton-flatoff ) . Fig 2 shows two typical 
zero-level patterns, and also their difference. This differ- 
ence is ^ 150 ADU with variations of ^ 50 ADU from one 
row to another, which induces, if not accounted for, a pho- 
tometric zero-point variation on the field at a few percent 
level. The zero-level has been measured using appropri- 
ate images {special flaton-flatoff frames), as suggested by 
C. Lidman (private communication, now fully described 
in the SOFI manual). In the determination of the zero- 
level it is assumed that during the observations the lamp 
is stable; this assumption has been verified a posteriori, by 
comparing of a few series of such measurements. 

In accurately reduced images the fiux of a standard star 
should not depend on its location. In order to test the 
accuracy of the flat-fielding, a standard star has been ob- 
served 16 times, displacing the telescope pointing between 
each exposure by ~ 80 arcsec, in such a way that the 
standard star appears on a grid 4x4, and its flux at the 
different positions on the chip has been measured. Two 
more stars were in the field of view and were also used 
for this test. Figure 3 shows the deviation from an ideal 
response (i.e. 1 everywhere), after gain correction and be- 
fore any other correction. The measured flux is well within 
1 % from 1 in most of the sampled locations. The RMS 
deviation is 0.7 % which, if not corrected for, would in- 
duce a 0.007 mag error. Part of the observed scatter is 
due to photometric errors on individual magnitude mea- 
surements, accounting for 0.005 mag. In just one location 
the measured flux differs by 2.4 %. However, this outlier 
value could be partly due to a transient slightly hot/cold 
pixel un-recognizcd as such and un-corrected for. We re- 
call that transient hot/cold pixels are recognized and cor- 
rected in the next phase, which combines images taken at 
different pointings. 

Since the RMS deviation from the mean is ^ 0.7%, our 
images do not require a supplementary illumination cor- 
rection, which is instead often considered in the reduction 
of near-infrared images. As a comparison, the RMS de- 
viation from the mean of the NICMOS 1 & 2 on Hubble 
Space Telescope is - 2% (Colina, Holfeltz & Richie, 1998) 
and no illumination correction is included in the NICMOS 
pipeline reduction, to our best knowledge. 

2.1.2. Background removal 

For the background subtraction a user-friendly software 
tool. Eclipse (Devillard, 1997), has been used. 

In the near-infrared, observers are faced with changes of 
the intensity, spectrum and spatial shape of the sky. Nod- 
ding the telescope modulates the astronomical objects in- 
tensity (on a give pixel) more quickly than the background 
variation. 

From a technical point of view, a high-pass filter will 
remove the slower changing background and will leave un- 



touched the higher frequence astronomical signal. To make 
this operation effective, Eclipse scales time-adjacent im- 
ages to the mean (or median) of the image that must be 
background-subtracted. In doing this operation. Eclipse 
assumes that the variation of the background is multiplica- 
tive and coherent over the whole field during the consid- 
ered short time interval. Then, images are low-pass fil- 
tered in the time-line direction, using a filter 11 exposures 
large centered on the image whose background is to be 
subtracted, allowing a rejection of the faintest and of the 
three brightest pixels in order to take into account the ex- 
istence of celestial objects, cosmic rays and intermitting 
hot/cold pixels. With this choice of the Eclipse settings, 
more bright pixels than faint ones are clipped during the 
sky determination. This is the natural choice when, as 
in our case, sky pixels contaminated by sources have an 
asymmetric intensity distribution, i.e. when the number 
of sources with positive counts exceeds the number cold 
pixels. While pixel masking would be preferable, it is not 
available within the Eclipse software. 

This operation gives the background which is subtracted 
to the original fiat- fielded image. We want to stress that 
no multiplicative scaling is applied to the image to be 
background-subtracted. We verified, by means of Midas 
pipelines developed by the author for the analysis of near- 
infrared images of the Coma cluster (Andreon et al. 2000) , 
that Eclipse correctly performs this complex task. 

2.1.3. Photometric calibration 

The airmass coefficient is computed by means of the sci- 
ence frames, due to the fact that AC 118 has been observed 
at several hour angles. We determine the airmass coeffi- 
cient from the measure, on 190 out of 265 frames, of the ap- 
parent flux of a small galaxy (a possible early-type galaxy 
of AC 118) in an uncrowded region. The upper panel of 
Figure 4 shows the airmass dependence of the instrumen- 
tal magnitude of the considered galaxy. The adopted value 
for the atmospheric absorption, 0.08 mag airmass"^, is 
compatible with that derived from the observations of the 
(admittedly few, 5) photometric standard stars observed 
during the night, and is equal to the value assumed by 
Persson et al. (1998) in defining standard magnitudes for 
the standard stars used here. Notice that the residual 
scatter in the data, after correction for airmass differential 
extinction, is 0.03 mag for the considered galaxy, of which 
0.03 mag (i.e. almost all) are due to photometric errors 
in individual magnitude measures. The similarity of the 
expected and observed scatters gives a direct confirmation 
that the observing night was photometric, and that the 
data reduction is accurate. 

Aperture magnitudes have been used to measure the 
apparent fiux of the standards, using the same aperture 
adopted by Persson et al. (1998) for the photometric stan- 
dard stars, i.e. 10 arcsec in diameter. The zcropoint scat- 
ter derived from standard stars is 0.008 mag, of which 
0.005 mag are due to photometric errors associated to the 
Persson et al. (1998) photometry. 

2.1.4. Combining the images and final details 

Images must be coaligned before coadding. The jitter 
data reduction procedure uses cross-correlation techniques 
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in the image domain to measure offsets witli subpixel pre- 
cision, adopting, as first estimates, the telescope spatial 
offsets written in the header of the FITS files. Images have 
been combined using the task imcombine under IRAF, 
since Eclipse does not fit our needs. Files containing the 
flux scaling (since AC 118 has been observed through dif- 
ferent airmasses) , the bad pixel mask and the relative spa- 
tial offsets were given in input. The task gives in output 
the composite image, the image of the measured dispersion 
among the input images and the number of pixels used (i.e. 
the exposure map in units of 1 minute). As "mean" for 
the imcombine task, we adopt a straight average allowing 
the rejection of up to one high and one low pixel values, 
to allow for the presence of cosmic rays or intermittent 
hot/cold pixels. We remember that imcombine deals only 
with integer shifts, and for this reason spatial offsets are 
rounded to the nearest integer. This fact inflates the final 
PSF by - 0.5 pixels. 

Finally, a bright saturated star produces on the final im- 
age a row brighter than average (and a symmetric ghost 
row in the opposite image quadrants). A constant, com- 
puted as described in Andreon (1993) has been subtracted 
to these two rows. 

The sky brightness ranged from 12.9 to 13.2 mag 
arcsec^^ during the night. The seeing, in the combined 
image, is 0.75 arcsec (FWHM, ~ 2.5 pixels). The sky 
noise in the fully exposed part of the image is 24.0 mag 
arcsec"^. 



2.2. Control field: Hubble Deep Field South 

Public images for the Hubble Deep Field South (HDF- 
S) have been taken in Ks with SOFI at NTT (Da Costa 
et al. 2000), i.e. with the same instrument, filter and 
telescope as the AC 118 image. This ensures an almost 
perfect homogeneity of the data. A few characteristics of 
HDF-S images are listed in Table 1, together with those 
relative to AC 118 observations. Public images are already 
reduced. The reduction of these data follows the same lines 
described for AC 118 and makes use in large part of the 
same software used by us, i.e. Eclipse. The claimed pho- 
tometric calibration has an error of 0.1 mag for HDF-S- 1 
and 0.05 mag for HDF-S-2 (Da Costa et al. 2000). 

Differently from AC 118 images, HDF-S images have 
been spatially resampled and filtered during the combining 
phase, and for this reason the noise in the final image looks 
smaller than it really is. As a consequence, magnitude er- 
rors in Da Costa et al. (2000) are underestimated. In order 
to perform an uncorrelated measurement of the noise, as 
we did for AC 118 images, we binned the HDF-S images, 
thus reducing the correlation between adjacent pixels. Ta- 
ble 1 presents the fully corrected noise in AC 118, HDF- 
S-1 and HDF-S-2. Once corrected for correlated noise, 
the noises in the HDF-S and AC 118 images are consistent 
with the exposure times. Galaxy counts computed from 
these images agree with literature ones (Da Costa et al. 
2000), as confirmed also by Figure 5. 



exposure time across the field of each image, due to the 
dithering, we consider here only the central square areas 
listed in Table 1. 

Galaxies have no well defined edges, therefore their lu- 
minosity depends on how galaxy edges are defined. We 
adopt Kron magnitudes (see Kron 1980 for the exact def- 
inition, and Berlin & Arnouts 1996 for a software imple- 
mentation) , defined as the fiux integrated in an area with 
size adapted to each galaxy. Unfortunately, Kron magni- 
tudes depend sensibly on the determination of the object 
size, which is very difficult for faint objects and in crowded 
regions, such as the core of AC 118. Therefore, aperture 
magnitudes are adopted for faint objects. In detail, as 
a measure of the magnitude of the galaxies, magnitudes 
computed within 2.5 Kron radii are adopted for galaxies 
brighter than Ks = 18 mag and aperture magnitudes (at 
the 4.4 arcsec aperture, which correspond to 24 Kpc) are 
used for fainter galaxies. Figure 5 shows that the two 
quantities give almost identical galaxy counts in a large 
magnitude range, and in particular near the bridge mag- 
nitude [Kg = 18 mag). Since the LF is computed by using 
galaxy counts, and not individual object magnitude, it is 
insensitive to differences between Kron and aperture mag- 
nitudes of each individual object. 

In computing galaxy counts it is a standard practice to 
correct them for missing flux, i.e. for the flux not within 
the isophote or aperture. Usually, this correction is ap- 
plied to all galaxies, independently on their luminosity. 
There is no reason for applying this correction for the LF 
determination, and furthermore, this correction has two 
shortcomings in our case. First of all, the absolute mag- 
nitude distribution of the galaxies of the same apparent 
magnitude differ in the two compared directions, because 
the sample in the cluster line of sight is the superposition 
of a volume complete (the cluster) and a flux limited sam- 
ple (the fore-background) while the control field is a flux 
limited sample. Thus, the luminosity correction is not the 
same, even at a fixed apparent magnitude, due to both 
cosmological effects (cosmological dimming and K correc- 
tion at least) and the surface brightness profile of galaxies 
which depends on the (unknown) absolute luminosity. Sec- 
ond, even under the optimistic hypothesis that corrections 
are perfectly known for each individual object, the catalog 
completeness is undermined by bright galaxies with low 
surface brightness (a long and thorough discussion on this 
topic is presented in Andreon & Cuillandres, 2000). 

The magnitude completeness is defined as the magni- 
tude where objects start to be lost because their central 
brightness is lower than the detection threshold. It is mea- 
sured as the brightest magnitude of the detected galaxies 
having central brightness equal to the detection thresh- 
old (see Garilli, Maccagni & Andreon 1999 for details). 
For AC 118, the (5(t) limiting magnitude is Ks ^ 20.5 
mag in a 4.4 arcsec aperture. Of course, many fainter and 
smaller objects are visible on the image, because the limit- 
ing magnitude is fainter using a smaller aperture, such as 
the "standard" 3 arcsec aperture, or a 3 FWHM aperture. 



2.3. Detection and completeness magnitude 

Objects has been detected by using SEx version 2.1 
(Bertin & Arnouts 1996). For AC 118 we made use of 
the RMS map for a clean detection. Due to the varying 



2.4. Star/galaxy classification 

Most previous similar studies have near-infrared images 
whose resolution is too coarse for allowing object classifi- 
cation as star or galaxy from the extent of the sources. 
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Thus, observers were obliged to adopt the classification 
performed either on optical images of the cluster (e.g. 
Barger et al. 1996) or colors (e.g. De Propris et al. 1999). 
Given the good seeing and sampling of the AC 118 Ks 
image, the star/galaxy classification can be based on the 
object extent, as measured in the near-IR image itself, by 
adopting the SEx star/galaxy classifier. The very central 
area of AC 118 has been observed by the Hubble Space 
Telescope (Couch et al. 1998). Only a few stars (as classi- 
fied by their compactness in the HST image) are brighter 
than the Kg completeness limits (AC 118 is at high Galac- 
tic latitude, see Table 1), and all of them are correctly 
classified as stars using the Kg image. Only a few galax- 
ies, out of hundreds, are misclassified as stars because of 
their compactness. Therefore, iJST confirms the goodness 
of our ground-based star/galaxy classification. 

2.5. Comparison to literature data 

AC 118 has been observed in the K' band by Barger et 
al. (1996). Figure 6 shows the comparison between their 
K' magnitudes, computed in a 5 arcsec aperture, and our 
Kg niag, computed in a 4.4 arcsec aperture for common 
objects. The line shows the "one-to-one" relation, and it 
is not a fit to the data. The agreement is good. Inspection 
of our image shows that outliers to the right of the "one- 
to-one" relation are pairs of objects, blended in Barger 
et al. (1996) and resolved here as separate objects, due 
to better seeing and sampling. A few of these objects are 
shown as inset in Figure 1. 

3. THE AC 118 LUMINOSITY FUNCTION 

The final image of AC 118 shows features which likely 
are gravitational arcs (Y. Mellier, private communication), 
some of which are previously unknown. The brightest two 
are magnified in the low panel of Figure 1. Other sus- 
pected lensed galaxies, not visible on the HST image be- 
cause they fall outside its field of view, are likely present, 
but their confirmation requires a full lensing analysis of the 
image, which is outside our aims. The arcs visible in the 
near-infrared image and the other visible on the HST im- 
age point out two main mass concentrations: a central one, 
and another one offset to the North- West, thus confirming 
the binary structure suggested by the spatial distribution 
of the galaxies (Figure 1). Inspection of public Rosat HRI 
& PSPC images shows a similar binary structure, whose 
brighter clump is coincident with the assumed center of 
AC 118, while the second clump is at the North-West. 

Figure 7 shows galaxy counts toward HDF-S and 
AC 118. Magnitudes have not been corrected for Galactic 
extinction, because the correction is very small (<^ 0.01 
mag and in any case negligible with respect to the HDF-S 
photometric zero point error). Counts in the cluster direc- 
tion are larger than in the field direction and the difference 
is large. 

The cluster LF is the statistical difference between 
counts in the cluster direction and in the control field direc- 
tion (Zwicky 1957, Oemler 1974). Its error should account 
for Poissonian fluctuations of counts along the cluster and 
field lines of sight and for non-Poissonian fluctuations of 
the counts, due to the non-zero galaxy correlation func- 

^For the time being, we use the term "dwarf for galaxies whose 



tion. We take into account the last term according to 
Huang et al. (1997). Thus, the statistical significance of 
any claim on the LF does not assume that fore/background 
in the cluster line of sight is the "average" one (or that 
observed in the control field), but instead takes into full 
account that the background counts fiuctuate, from re- 
gion to region, and fluctuate more than ^Jn. We stress 
that background fluctuations enter twice in the error bud- 
get because the LF is given by a difference of two galaxies 
counts, each one subject to background fluctuations. For 
simplicity, we assume that errors of different nature can be 
added in quadrature (i.e. that standard error propagation 
laws can be used) , and in the error propagation we neglect 
the difference between Poissonian and Gaussian distribu- 
tions. 

In the very center of the cluster, galaxy density is high. 
Thus, in order to perform an accurate measure of the LF 
in this area, a crowding correction must be computed. For 
each magnitude bin, we consider as the area usable for 
the detection the one which is not filled by brighter galax- 
ies. Since galaxies have no well defined edges, we assign to 
each galaxy an excluded area equal to half their isophotal 
{p ^ 24 mag arcsec"^) area. This correction turn out to 
be significant only in the central region of AC 118. Re- 
sults are robust with respect to the implemented crowding 
correction. 

Figure 8 shows the AC 118 global LF, i.e. those com- 
puted using all the ~ 5 x 5 arcmin field of view, correspond- 
ing to 2.66 Mpc^ at the AC 118 redshift. The best fitting 
Schechter (1976) function is also plotted. In order to take 
into account the amplitude of the bin, we fit the data with 
a Schechter function convolved with the bin width. The 
best fit parameters are: X* = 15.3 mag {Mk* — —26.0 
mag) and a = —1.2, where a is the slope of the faint part 
of the LF, and M* is the knee of the LF, i.e. the magni- 
tude at which the LF starts to decrease exponentially. The 
confidence levels of the best fit Schechter parameters are 
plotted in Figure 9. The AC 118 global LF is smooth, well 
described by a Schechter function (xj = 0.7) and has an 
intermediate slope, down to Kg = 20.5 mag {Mk^ = —20.8 

mag), which is ^ 5 mag below M* , well in the dwarf regime 

1 

Figure 10 summarizes two relevant features of the pub- 
lished cluster LFs in the near-infrared: it shows the mag- 
nitude limit reached as a function of the area surveyed. 
Points in the lower-right corner of the diagram are the 
most informative about the LF, because of the large area 
coverage and of the depth reaching the dwarf luminosity, 
but they are also the most expensive in terms of telescope 
time. The present study is marked by the point nearest to 
the lower-right corner of the graph. With respect to previ- 
ous determinations, we explore at the same time one of the 
largest area and one of the deepest LFs, mainly because 
SOFI has 16 times more pixels, on average, than many 
previous instruments and because observations have been 
tailored for Fundamental Plane studies, which require to 
measure the surface brightness profile of galaxies, not just 
their integrated magnitude. Thus, present data offer the 
possibility to study the environmental dependence of the 
LF and to sample possible effects on dwarf galaxies, in 
contrast to previous near-infrared cluster surveys that ex- 
magnitude is M >~ M* + 4 
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M* is brighter in the outer region and therefore bright 
galaxies are more abundant there, in apparent contradic- 
tion with our previous claims. However, errors on best fit 
parameters are strongly coupled and one should be cau- 
tious when interpreting changes in one parameter, say M* , 
when the two other parameters (a and (jf) change at the 
same time. Since parameters are coupled, we emphasize 
the direct comparison of the data itself. 

The excess of dwarfs in the outer region is large enough 
to make the number of dwarfs per typical galaxy in this re- 
gion larger than the one observed for the global LF (which 
includes the outer region): we found 2.8l]^'5 more dwarfs 
here than over the whole area. With respect the SW re- 
gion, the figure is 2.6l^'5. 

In conclusion, the near-infrared luminosity function of 
AC 118 depends on the considered cluster location: in the 
inner central region, dense in galaxies, there are more 
bright galaxies and less dwarfs per unit typical galaxy, 
than in the NW clump or in the outer region. In other 
words, the AC 118 LF steepens going from high to low 
density regions. The outer region is the richest, among 
the three considered, in dwarfs and, at the same time, 
the poorest in very bright galaxies. The found differences 
among LFs measured in different cluster regions cannot be 
due to variations of background counts among the various 
cluster lines of sight because we have fully included in the 
error budget this source of error (which is the largest one). 

3.1. Comparison to literature LF 

Barger et al. (1996) present the K' (virtually indistin- 
guishable from Kg for our purposes, see Figure 6) com- 
posite LF of the very central region of three intermediate 
redshift clusters at z ~ 0.3, one of which is AC 118. Their 
LF reaches K' = 19 mag, i.e. 1.5 mag brighter than our 
magnitude limit (compare B96 and our points in Figure 
10). They have in their composite sample ~ 300 member 
galaxies to be compared to the ~ 500 cluster galaxies of in 
our sample of AC 118 alone. Their best fit parameters are 
K' ^ 15.74±0.13maganda = -1.0±0.12. Since Barger 
et al. (1996) observed only the central region of the clus- 
ter, we consider here only the central area of AC 118 and 
we compare their best fitting values to those we derived for 
the central clump (which of course includes just a fraction 
of our total sample of galaxies) . We computed confidence 
levels for the best fit of the composite LF of Barger et 
al. (1996) reading their data and errors from their Figure 
7. Figure 15 presents our confidence levels for the main 
clump of AC 118 and for the composite sample of Barger 
et al. (1996). The two Icr confidence levels largely overlap, 
thus meaning that the two LFs are equal within the errors. 
This is expected, due to the good agreement between in- 
dividual magnitudes of galaxies in common shown in Fig. 
6 2. 

The Ks-band tightly maps the rest-frame i/-band 
emission for objects at z ~ 0.3. Thus, without making 
any assumption on the K-correction value, we can com- 
pare the H band zero-redshift LF of the Coma cluster (De 
Propris et al. 1998, Andreon & Pello 2000) to our Kg band 

^Barger et al. (1996) quote errors of the best fit parameters without specifying for how many free parameters they are. The comparison of 
our Fig. 15 to their errors shows that their quoted errors are for one free parameter. Adopting the usual convention of quoting error for the 
number of interesting parameter of the fit (see e.g. Avni 1976 or Press et al. 1993), which in the present case is two, i.e. a and M* , the actual 
error on the slope of the LF is about twice larger than claimed by Barger et al. (1996). 



plored either smaller areas or a smaller magnitude range. 

In order to search for a dependence of the LF on the en- 
vironment, we compute the LF in two regions, centered on 
the two clumps of AC 118 (see Figure 1). Each region has 
an area of ~ 0.5 Mpc^, i.e. the typical area observed at 
comparable depth (see Figure 10). The two LFs are plot- 
ted in Figure 11, together with their best Schechter fits. 
A Kolmogorov-Smirnov test shows that data points differ 
at 99.5 % confidence level (i.e. they differ at ^ 3cr, but it 
should be remembered that this test makes no use of er- 
rors). The main clump of AC 118 is poorer in dwarfs and 
is richer in bright galaxies than the North-West clump. 
Figure 12 shows the confidence levels of the best fitting 
Schechter parameters. Assuming that the data are drawn 
from a Schechter function, they differ at almost 95 % con- 
fidence level (i.e. ^ 2a, but now errors are taken into 
account). This plot confirms the previous finding: a is 
shallower in the main clump than in the NW clump, in- 
dicative of an environment poor in dwarfs. 

We now compare the LF computed in the main clump 
of AC 118 with that in the outer region. The outer region 
excludes both the two ~ 0.5 Mpc^ regions centered on 
the two clumps and also a no-man-land around the NW 
clump (see Figure 1). The galaxies in the main clump 
have an average projected distance of 180 kpc, while the 
galaxies in the outer region are 580 kpc away, on aver- 
age, and 1.1 Mpc at most. Thus, the outer region is still 
well inside the cluster, as confirmed also by the fact that 
the galaxy overdensity is large enough for the LF to be 
computed with the differential counts method. Figure 13 
shows that the inner and outer LFs differ (at more than 
99.99999 % confidence level according to a Kolmogorov- 
Smirnov test): there are far more dwarfs and somewhat 
fewer bright galaxies per typical galaxy (say Kg ~ 18 
mag) in the outer region. In order to quantify these ex- 
cesses, galaxies having K < 17 ,17 < K < 1% , K > 19 mag 
are defined bright, typical and dwarf galaxies, respectively. 
These break magnitudes correspond, at the cluster red- 
shift, to Mk = -24.2 and Mk = -22.2 mag, respectively. 
We found that the number of dwarfs per typical galaxy in 
the outer region exceeds the one observed in the central re- 
gion by a factor 5.3^23; where errors are quoted at 99.9% 
confidence level and are computed according to Gehrels 
(1996), i.e. by taking into account that the error on this 
ratio is partly binomial and partly Poissonian. The excess 
of bright galaxies in the main clump, with respect to the 
NW region, is instead 2.61^;^ (99.9% confidence level). 

Figure 14 gives the confidence levels of the best fitting 
Schechter parameters for the inner and outer region. They 
are located in quite different parts of the diagram and 
point out the same differences shown in Fig. 13. The 
same figure also shows that K* is undetermined in the 
outer region. In other terms, the outer region LF can be 
accurately described by a powerlaw. 

There are some caveats in comparing values of one single 
best fit parameter, say M*, when they are computed by 
means of a fit with a function with three free parameters: 
a superficial inspection of the plotted confidence contours 
or of Table 2 would lead to a fairly different conclusion: 
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LF of AC 118. 

The needed transformation is: 

Mh^^o = -{m-M)+ if ,=0.3 + 0.09 

The two Coma LFs are measured on different parts of 
the Coma cluster: Dc Propris et al. (1998) studied a ^ 1 
Mpc^ area around the cluster center and found a shallow 
slope {a = —1.0), with an hint of steepening at faint mag- 
nitudes. Andreon & Pello (2000) instead studied a ~ 0.6 
Mpc^ area ofFcentered by ^ 0.4 Mpc and found an overall 
slope of a = —1.3. Both works found M'^ ~ —24.6 mag. 
For AC 118, we found K* = 15.3 to 16.8 mag depending 
on the considered area, which corresponds to M^ = —26.2 
to —24.7 mag, in good agreement with Coma M^ when 
errors are taken into account. The trend for a LF shal- 
lower in the central region than in the other more exter- 
nal region suggested for Coma by Andreon & Pello (2000) 
is confirmed for AC 118, with the major difference that 
data and analysis are homogeneous for the latter cluster, 
whereas for the former are not. 

Figure 16 compares the LF computed in the outer re- 
gion of AC 118 to the field LF, measured at 0.2 < z < 0.6 
(Cowie et al. 1997). These LFs are measured in two very 
different ways: the cluster LF is computed on a volume- 
limited sample, whereas the field LF is computed on a 
flux-limited sample (in fact, the Cowie et al.' sample in 
not actually flux-limited, because it is incomplete and has 
a complex selection function). The two LFs, both con- 
cerning galaxies at intermediate redshift, have very similar 
shapes, and both conspicuously differ from the LF of the 
inner region of AC 118 shown in Figure 13. The similarity 
of these two LFs corroborates both the change of the LF 
parameters with the environment and the measured shape 
of field LF, which is based on a sample with a complex 
selection function. Environment seems to have played no 
role in modifying the near-infrared LF in the outer re- 
gion of AC 118, because of the similarity of the LFs in 
the field and in the AC 118 outer region. Instead, Figure 
13 shows that the environment modifies the LF shape at 
larger galaxies density or smaller clustercentric distances. 

4. DISCUSSION 

The main result of this study is that the LF shape de- 
pends on location within the cluster. Among the studied 
regions, the cluster center, i.e. the main clump {d ^ 180 
kpc), has many more bright galaxies per unit typical 
galaxy (say Ks ~ 18 mag, Mh ~ —23.5 mag), as shown 
in the previous section. The converse holds for dwarfs, 
which are less numerous in the center than elsewhere. The 
other observed clump, the second one in X-ray luminos- 
ity, presents LF characteristics intermediate between the 
central clump and the outer region. At an average clus- 
tercentric distance of ^ 580 kpc, the AC 118 LF is statis- 
tically indistinguishable, whithin the present errors, from 
the field LF at similar redshift. 

Our definition of the main clump is sound: X-ray images 
and gravitational lensing distorsion of background galaxies 
point out two main concentrations, of which the brightest 
in X-ray is also the one which we call main. Thus, our def- 
inition of main and center are independent and therefore 
unbiased by the presence of bright galaxies. 

There is not anything like a near-infrared LF of AC 118, 
since it depends on the surveyed region. A similar result 



has been suggested for the Coma cluster by Andreon & 
Pello (2000) from the comparison of heterogeneous near- 
infrared LFs computed by different authors on different 
portions of the Coma cluster. In the optical R band, 
Driver, Couch & Phillipps (1998) and Seeker, Harris & 
Plummcr (1997) find a large variation in the ratio of dwarf 
to giant galaxies as a function of the clustercentric dis- 
tances, with dwarfs more numerous at large (r > 0.56 
Mpc) distances (or lower density regions). This trend is 
in agreement with the dependency of the slope of the LF 
with the density in the ^ 65 clusters studied by Garilli, 
Maccagni & Andreon (1999). Due to the small Hubble 
Space Telescope field of view, it is not possible to measure 
whether the found variation of the LF is due to a change 
in the relative number density of galaxies of each mor- 
phological type (i.e. LFs of the morphological types are 
universal) or rather to luminosity changes of the individual 
galaxies (in that case the LFs of each morphological type 
depend on location), and should be deferred until Hubble 
Space Telescope images for a larger field will be hopefully 
taken. In the optical, the bright part of the luminosity 
functions of the morphological type are found not to de- 
pend on the considered environments and differences in 
the optical LF are found mainly due to differences in the 
cluster morphological composition (Binggeli et al. 1986, 
Jerjen & Tammann 1997; Andreon 1998). 

The spatial dependence of the AC 118 near-infrared 
LF implies a luminosity segregation: bright galaxies are 
found preferentially in the cluster center. There are sev- 
eral claims (e.g. Driver, Couch & Phillipps 1998; Seeker, 
Harris & Plummer 1997) of a luminosity segregation for a 
few rich clusters, but this evidence is restricted to optical 
bands. In this paper, this evidence is extended to a near 
infrared band. Since the near-infrared luminosity traces 
stellar mass (Bruzual & Chariot 1993), this result implies 
a mass segregation more tightly than under the usual as- 
sumption than optical luminosity traces mass: here we 
show directly that massive galaxies are found preferentially 
in the cluster center. Thus, more massive clumps arc more 
tightly bound to the clusters, which is a general outcome 
of the simulations of a hierarchical universe (Kauffmann 
et al. 1997). The hostile cluster environment plays a role 
in shaping the AC 118 LF but only at small clustercentric 
radii (or high density), since the outer region AC 118 LF 
is quite similar to the field FL. 

A possible not-ubiquitous cluster LF, i.e. varying with 
cluster radius, implies a dependency of the LF parame- 
ters {M* , a) on the surveyed area and suggests caution 
in performing cosmological tests involving M* as a stan- 
dard candle, or in studying the galaxy evolution through a 
change in the best fit LF parameters. Recently, De Propris 
et al. (1999) report a brightening of M* with increasing 
redshift for their cluster sample, which is characterized 
by a surveyed area (in Mpc^) varying with redshift in a 
complex way due to the variety of field of views of used 
instruments and sampled redshifts. They average a few 
cluster at each redshift, in order to reduce the impact of 
a not-ubiquitous cluster LF. Typically, areas range from 
1.5 to 3.5 Mpc. However, the problem of a not-ubiquitous 
LF cannot probably be circumvented by choicing a fixed 
area in the cluster rest-frame because clusters have not 
standard size and shapes. Given the importance of de- 



Deep Ks luminosity function at z=0.3 



termination of the evolution of M*, we consider useful to 
check on a large sample of clusters the claim of a possible 
not-ubiquitous cluster near-infrared LF. 

This work is part of a collaboration with M. Arnaboldi, 
G. Busarello, M. Capaccioli, G. Longo, P. Merluzzi and G. 
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D. Maccagni for a careful reading of this manuscript and 
useful suggestions. A special thank goes to the anony- 
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Table 1 
The data 



AC 118 HDF-S-1 HDF-S- 2 



265 


180 


300 


0.75 


0.90 


0.96 


24.0 


24.0 


24.2 


20.5 


20.5 


21.0 


-81 


-49 


-49 



Exposure time (min) 

Seeing (FWHM, arcsec) 

Fully corrected noise'' (mag arcsec^^) 

Complet. mag {(f) = 4. 4) 

Galact. lat. (degree) 

Galact. abs. E{B - Vf 0.013 0.028 0.028 



Note. — The full SOFI field of view is 5 x 5 arcmin. The used field of view 
is4'52"x4'52. 

" Color excess has been measured using COBE/DIRBE maps (Schlegel, 
Finkbeiner & Davis 1998). Galactic absorption in Ka is Ak ~ 0.2E{B — V) 
(Allen, 1955) 

The sky noise is measured as the dispersion of adjacent pixels in the back- 
ground, once the image is binned in pixels of 1 arcsec for reducing the corre- 
lation between adjacent pixels in the HDF-S images. 



Table 2 
Best Fit values 



Region K* a (p x 



2 



Global 


15.32 


-1.18 


31.21 


7.21 


Main clump 


16.59 


-0.47 


30.74 


4.33 


SW clump 


17.02 


-0.91 


19.24 


10.05 


outer region 


unconstrained 


-1.69 


0.21 


10.01 



Note. — All fits have 9 degrees of freedom and three 
free parameters, of which two {K* and a) are interesting. 
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Fig. 1. — The Kg band image of AC 118. The field of view is ^ 5 x 5 arcmin. North is up and East is to the left. The 
large boxes delimit the main and North- West regions (large square and rectangle, respectively). The "outer region" (see 
text) is the area below the slanted line and outside the large square. Small boxes mark the objects blended in Barger et 
al. (1996) image (the 6 small boxes at East) or gravitational lenses (the two small boxes at West). All them are magnified 
in the lower panel. The field of view of each zoom is 15 arcsec on side. 
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Fig. 2. — Upper panel: zero-level of flaton (solid line) and flatoff (dotted line). Spikes are due to bad pixels, and do not 
reflect real variations of the zero-level. Lower panel: difference of the two zero-levels. 
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Fig. 3. — Illumination correction. Values indicate the deviation from a perfect flat-fielding (i.e. from 1 everywhere) and 
are located where measurements were performed. Typical measurement errors are 0.005 
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Fig. 4. — Upper panel: Airmass dependence measured by a possible elliptical galaxy of AC 118. Lower panel: instrumental 
magnitude, airmass corrected, of the same galaxy. Notice the small scatter (0.03 mag) almost entirely due to photometric 
errors on the single measure. 
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Fig. 5. — Galaxy counts adopting Kron (open points) and aperture (filled points) magnitudes in the HDF-S line of sight. 
Counts are identical almost at all magnitudes, and in particular at the bridge magnitude Ks — 18 mag. The hashed 
region delimits the locus occuped by literature galaxy counts in the K band, as estimated from the compilation presented 
in Figure 1 of McCracken et al. (2000). The large amplitude of this region is due in part to heterogeneity of the data and 
reduction of the compared works. 
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Fig. 6. — Comparison between Barger et al. (1996) K' magnitude in a 5 arcsec aperture with our Ks magnitudes in a 
4.4 arcsec aperture for common objects. The Hne is the "one-to-one" relation, not a fit to the data. Outliers to the right 
of the line (marked with a circle) are pairs of objects, blended in Barger et al. (1996) image but resolved as separated 
components in our image. The horizontal dotted line marks the Barger et al. (1996) catalog limit. 
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Fig. 7. — Raw galaxy counts in the HDF-S fields and toward AC 118. Errorbars are simply taken as y/n. 
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Fig. 8. — AC 118 global luminosity function. Errorbars are ±1ct and take into account both Poissonian and non-Poissonian 
fluctuations, i.e. include the cosmic variance of background galaxy counts. Both apparent and absolute magnitude scales 
are presented, as in most of the following figures. 




Fig. 9. — 68 % and 95 % confidence levels of the best fitting Schechter parameters for the global AC 118 LF. The filled 
square marks the best fit. 
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Fig. 10. — Rest-frame surveyed area vs. absolute magnitude limits for previous determination of the cluster near-infrared 
LF. LFs presented in Aragon-Salamanca et al. (1993), Barger et al. (1998), Barger et al. (1996), dc Propris et al. (1998; 
1999), Trentham & Mobasher (1998), Andreon & Pello (2000) are denoted as AE93, B98, B96, dP98, dP99, TM98 and 
APOO, respectively. The characteristic magnitude, M*, of the Schechter LF is also marked. The line connect two different 
LF determinations of the Coma LF. 
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Fig. 11. — AC 118 LF of the two clumps. Filled squares (open circles) mark the LF computed for the main (NW) clump. 
Errorbars are as in Figure 8. The two curves arc the best Schechter fits (solid line for the main clump, dotted line for the 
NW clump). 
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Fig. 12. — 68 % and 95 % confidence levels of the best fitting Schechter parameters for the main clump (solid lines) and 
NW clump (dotted lines) of AC 118. The filled square and the open circle marks the best fit values. 
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Fig. 13. — AC 118 LF of the inner and outer region. Solid squares (open circles) mark the LF computed for the inner 
(outer) region. Errorbars are as in Figure 8. The two curves are the best Schechter fits (solid line for the inner region, 
dotted line for the outer region). 
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Fig. 14. — 68 % and 95 % confidence levels of the best fitting Schechter parameters for the inner region (solid lines) and 
outer region (dotted lines) of AC 118. The filled square marks the best fit of the inner region LF. The best fit for the 
outer region is undetermined. 




Fig. 15. — 68 % and 95 % confidence levels of the best fitting Schechter parameters for the inner region of AC 118 (solid 
lines) and the composite LF of 3 clusters at z ~ 0.3, including AC 118, presented in Barger et al. (1996) (dotted lines). 
The filled square and the open circle mark the best fit values. 
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Fig. 16. — LF in the outer region of AC 118 (open points) and in the field at 0.2 < z < 0.6 (from Cowie et al. 1996, 
filled triangles connected by a solid line). The field LF has been vertically shifted to match the AC 118 LF. 



